Computation systems based upon adaptive learning with fine-grained parallel architectures have moved out of obscurity in recent years because of the growth of computer-based information gathering, handling, manipulation, storage, and transmission. Many concepts applied in these systems represent potentially efficient approaches to solving problems such as providing automatic recognition, analysis and classification of character patterns in a particular image. Ultimately, the value of these techniques in such systems depends on their effectiveness or accuracy relative to conventional approaches.
In a recent article by Y. LeCun entitled "Generalization and Network Design Strategies," appearing in Connectionism in Perspective, pp. 143-155 (Elsevier Science Publishers: North-Holland 1989), the author describes five different layered network architectures applied to the problem of optical digit recognition. Learning in each of the networks was attempted on pixel images of handwritten digits via inherent classification intelligence acquired from the backpropagation technique described by D. Rumelhart et al., Parallel Distributed Processing, Vol. I, pp. 318-362 (Bradford Books: Cambridge, Mass. 1986). Complexity of the networks was shown to increase from a two layer, fully connected network called Net-2 to a hierarchical network called Net-5 having two levels of constrained feature maps for hierarchical feature extraction. The network Net-2 was said to have a significantly larger standard deviation in generalization performance than single layer, fully connected networks indicating, thereby, that the former network is largely undetermined with a large number of solutions consistent with its training set. But, as stated by LeCun, "[u]nfortunately, these various solutions do not give equivalent results on the test set, thereby explaining the large variations in generalization performance . . . it is quite clear that the network is too big (or has too many degrees of freedom)." Performance of the most complex hierarchical network, that is, Net-5, exceeded that of the lesser complex networks. Moreover, it was postulated that the multiple levels of constrained feature maps provided additional control for shift invariance.
While the hierarchical network described above appears to have advanced the art of solving the character recognition or classification problem, it is equally apparent that existing systems lack sufficient accuracy to permit realization of reliable automatic character recognition apparatus.